In this paper we look at some of the design issues that affect the success ofmultimodal displays that combine acoustic and haptic modalities. First, issuesaffecting successful sonification design are explored and suggestions are madeabout how the language of electroacoustic music can assist. Next, hapticinteraction is introduced in the light of this discussion, particularlyfocusing on the roles of gesture and mimesis. Finally, some observations aremade regarding some of the issues that arise when the haptic and acousticmodalities are combined in the interface. This paper looks at examples of whereauditory and haptic interaction have been successfully combined beyond thestrict confines of the human-computer application interface (musicalinstruments in particular) and discusses lessons that may be drawn from thesedomains and applied to the world of multimodal human-computer interaction. Theargument is made that combined haptic-auditory interaction schemes can bethought of as musical instruments and some of the possible ramifications ofthis are raised.
展开▼
